Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report unhappy blocks in happy path test #430

Merged
merged 16 commits into from
Sep 26, 2023

Conversation

al8n
Copy link
Contributor

@al8n al8n commented Sep 20, 2023

As mentioned in the meeting, we want to make sure in happy path tests, if a block contains AggratedQc, then we need to persist the block information.

I implemented a drop method for Block and feature gate it, this way we do not need to modify any of our current codebase, and we can easily remove the code and feature in the future when we fully solve the problem.

@al8n al8n added the enhancement New feature or request label Sep 20, 2023
@al8n al8n self-assigned this Sep 20, 2023
Comment on lines 12 to 35
impl<Tx: Clone + Eq + Hash, Blob: Clone + Eq + Hash> Drop for super::Block<Tx, Blob> {
fn drop(&mut self) {
let header = self.header();
// We need to persist the block to disk before dropping it
// if there is a aggregated qc when testing happy path
if let Qc::Aggregated(qcs) = &header.parent_qc {
#[derive(serde::Serialize)]
struct Info<'a> {
id: String,
view: View,
qcs: &'a AggregateQc,
}

let mut file = debug_file().lock();
let info = Info {
id: format!("{}", header.id),
view: header.view,
qcs,
};
// Use pretty print to make it easier to read, because we need the this for debugging.
serde_json::to_writer_pretty(&mut *file, &info).unwrap();
}
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be a simple method or function and not do it in drop. It is better to see the call explicetely it in the test.

Copy link
Contributor

@zeegomo zeegomo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with Daniel, but apart from that don't we already have the safe_blocks field in

pub safe_blocks: HashMap<BlockId, consensus_engine::Block>,
?

@al8n
Copy link
Contributor Author

al8n commented Sep 20, 2023

Agree with Daniel, but apart from that don't we already have the safe_blocks field in

pub safe_blocks: HashMap<BlockId, consensus_engine::Block>,

?

Yeah, we have safe_blocks, I previously thought we wanted to do the check for each block. Now, just use safe_blocks.

Comment on lines 11 to 39
pub fn persist(name: &'static str, blocks: impl Iterator<Item = &Block>) {
// We need to persist the block to disk before dropping it
// if there is a aggregated qc when testing happy path
if let Qc::Aggregated(qcs) = &block.parent_qc {
#[derive(serde::Serialize)]
struct Info<'a> {
id: String,
view: View,
qcs: &'a AggregateQc,
}

let mut file = OpenOptions::new()
.create(true)
.truncate(true)
.read(true)
.write(true)
.open(format!("{name}.json"))
.unwrap();
let infos = blocks
.map(|b| Info {
id: format!("{}", header.id),
view: header.view,
qcs,
})
.collect::<Vec<_>>();
// Use pretty print to make it easier to read, because we need the this for debugging.
serde_json::to_writer_pretty(&mut *file, &infos).unwrap();
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should persist them, here are the steps I think need to be done:

  • We can add a query for requesting the blocks to the node.
  • Before killing the nodes, retrieve blocks.
  • Check blocks with this function (or similar).
  • Report NodeId BlockId View for every block that is not starndardQc

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have an api to get blocks from the node, I think we can reuse that. In fact, it's the same that we use to check the view, it also has an additional field with blocks info

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have an api to get blocks from the node, I think we can reuse that. In fact, it's the same that we use to check the view, it also has an additional field with blocks info

Even better!

@zeegomo
Copy link
Contributor

zeegomo commented Sep 20, 2023

I previously thought we wanted to do the check for each block

What do you mean? safe_blocks contain all blocks, so we can check that all of them have an happy qc

@al8n
Copy link
Contributor Author

al8n commented Sep 20, 2023

I previously thought we wanted to do the check for each block

What do you mean? safe_blocks contain all blocks, so we can check that all of them have an happy qc

Ah, I think there may be some "intermediate" blocks, and we also want to check it, so I just implemented a drop method for it .

@zeegomo
Copy link
Contributor

zeegomo commented Sep 20, 2023

Ah, I think there may be some "intermediate" blocks, and we also want to check it

All intermediate blocks are in safe blocks apart from outright malicious / invalid ones

@danielSanchezQ danielSanchezQ changed the title Add supports to persist the block for testing happy path Report unhappy blocks in happy path test Sep 21, 2023
Copy link
Collaborator

@danielSanchezQ danielSanchezQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just missing the nodes id that have the unhappy block in the info. Otherwise this is it!!

Copy link
Contributor

@youngjoon-lee youngjoon-lee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! + I agree with #430 (review)

@al8n al8n merged commit 5b8fb85 into nomos-node-tree-overlay Sep 26, 2023
4 of 11 checks passed
@al8n al8n deleted the nomos-node-tests branch September 26, 2023 10:25
bacv added a commit that referenced this pull request Oct 5, 2023
* Use tree overlay in nomos node

* Use tree overlay in tests module

* Handle unexpected consensus vote stream end in tally

* Spawn the next leader first (#425)

* Report unhappy blocks in happy path test (#430)

* report unhappy blocks in the happy path test

* Make threshold configurable for TreeOverlay (#426)

* Modified test, so that all nodes don’t all connect to the first node only (#442)

* merge fix

---------

Co-authored-by: Youngjoon Lee <[email protected]>
Co-authored-by: Al Liu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants